Medical navigation system employing optical position sensing and method of operation thereof

ABSTRACT

An apparatus and method that uses shape sensing and imaging to record, display and enable return to an imaging probe location or to predetermined imaging parameters. The apparatus includes an ultrasound probe ( 104, 304, 404, 750 ); a shape-sensing-device (SSD) ( 102, 302, 602, 740 ) associated with the ultrasound probe; and a controller ( 122, 710 ). The controller may be configured to: determine at least one of location and orientation of the ultrasound probe based upon position sensor information (PSI) received from the SSD; select a view of a plurality of views of a workflow that are stored in the memory; obtain view setting information (VSI) including parameters and a position and/or orientation of the ultrasound probe for each of the views; determine guidance information; and render the determined guidance information on the rendering device and set ultrasound probe parameters based on the parameters of the VSI for the selected view.

CROSS-REFERENCE TO PRIOR APPLICATIONS

This application is the U.S. National Phase application under 35 U.S.C.§ 371 of International Application Serial No. PCT/EP2017/065946, filedon Jun. 28, 2017, which claims the benefit of U.S. Patent ApplicationNo. 62/356,566, filed on Jun. 30, 2016. This application is herebyincorporated by reference herein.

FIELD

The present system relates to a medical navigation system which employsshape-sensing methods such as optical-shape-sensing (OSS) methods toperform tracking of a surgical implement during surgical interventionsand, more particularly, to a medical navigation system which employsFiber Optic RealShape™ (FORS) tracking methods to track surgicalimplements during surgical interventions, and methods of operationthereof.

BACKGROUND

Finding an optimal position and view to image an anatomy of an objectsuch as a patient using ultrasound can be challenging especially duringprocedures such as surgical interventions. For example, duringprocedures such as structural heart disease procedures, atransesophageal echo (TEE) probe, which is a type of an ultrasoundprobe, may be placed such that a specific view of a heart of a patientmay be obtained. However, more than one view from the TEE probe may bedesired which may require a clinician to manipulate the TEE probe toother desired positions and/or orientations for these views and adjustultrasound parameters for each of these views. Unfortunately,manipulating the position and/or orientation of the TEE probe andvarying of the ultrasound parameters can take valuable time especiallyduring surgical interventions. Further, in addition to wasting time, itis often difficult, if not impossible, to return to a previous viewduring a procedure. Thus, even the best ultrasound clinicians can findit challenging to work with TEE probes during surgical interventions.

EP2289452A2 of Christopher Hasser et al., published Jun. 5, 2006,entitled “Laparoscopic Ultrasound Robotic Surgical System” is directedto a laparoscopic ultrasound robotic surgical system comprising a firstrobotic arm mechanically coupled to an ultrasound probe; a secondrobotic arm mechanically coupled to a surgery related device; a mastermanipulator; a control switch having user selectable first and secondmodes; and a processor configured to cause the second robotic arm to belocked in position and the first robotic arm to move the ultrasoundprobe according to user manipulation of the master manipulator when thecontrol switch is in the first mode, and cause the second robotic arm tomanipulate a tool according to manipulation of the master manipulatorand the first robotic arm to move the ultrasound probe according tostored instructions upon detection of a user command associated with thestored instructions when the control switch is in the second mode.

WO2015092581A1 of Bharat Ramachandran et al., published Jun. 25, 2015,entitled “Shape Sensed Robotic Ultrasound for Minimally InvasiveInterventions” is directed to systems and methods for shape sensedrobotic ultrasound for minimally invasive interventions. One or moremedical devices, such as, e.g., an ultrasound probe and endoscope, areintegrated with optical shape sensing. Shape sensing may be integratedwith the one or more medical devices by securing at least one fiber tothe one or more medical devices using, e.g., a sleeve, shrink tubing, achannel within the probe, patch attachment, etc. Based on the shapesensing data, a registration is performed between the one or moremedical devices. Registration may be, e.g., landmark-based,fixture-based, image-based, etc. In some embodiments, the one or moremedical devices are coupled to one or more moveable features of aconfigurable device or robot for robotic guidance. The one or moremoveable feature may also be integrated with shape sensing such thattheir relative positions are known.

US20140121489 A1 of Mohan Krishna Kommu CHS, published May 1, 2014,entitled “Medical Imaging System and a Portable Medical Imaging Devicefor Performing Imaging” is directed to a portable medical imaging devicefor performing imaging procedures. The portable medical imaging devicecomprises a processor configured to analyze location informationassociated with the portable medical imaging device. Then it isdetermined whether the location information is linked to a location ofperforming an imaging procedure. The processor selects an imagingconfiguration from a plurality of imaging configurations associated withan imaging procedure to be performed in the location. Each imagingconfiguration of the plurality of imaging configurations is associatedwith a type of imaging procedure. The plurality of imaging procedures isstored in at least one memory communicably coupled to the processor. Theselected imaging configuration is loaded to perform the imagingprocedure.

Accordingly, embodiments of the present system may overcome these andother disadvantages of conventional imaging systems.

SUMMARY

The system(s), device(s), method(s), arrangements(s), user interface(s),computer program(s), processes, etc. (hereinafter each of which will bereferred to as system, unless the context indicates otherwise),described herein address problems in prior art systems. Embodiments ofthe present system may employ shape sensing registration methods whichmay employ sample points, such as what may be considered a discrete setor a continuum of simultaneous sample points which may provideinformation related to a position of a coordinate system so thatdisparate coordinate systems may be registered.

In accordance with embodiments of the present system, there isdisclosed, in one aspect of the present system, a surgical guidancesystem including a memory; a rendering device; an ultrasound probe; ashape-sensing-device (SSD) associated with the ultrasound probe andhaving a predetermined position and orientation with regard to theultrasound probe, a controller coupled to the memory, the renderingdevice and the SSD, the controller configured to: determine at least oneof location and orientation of the ultrasound probe based upon positionsensor information (PSI) received from the SSD; select a view of aplurality of views of a workflow that are stored in the memory; obtainview setting information (VSI) for the selected view from the memory,the VSI comprising parameters and at least one of a position andorientation of the ultrasound probe for each of the views; determineguidance information based upon a comparison of the determined locationand orientation of the ultrasound probe and the at least one of aposition and orientation of the ultrasound probe for the selected view;and render the determined guidance information on the rendering deviceand set ultrasound probe parameters based on the parameters of the VSIfor the selected view.

In accordance with embodiments of the present system, the SSD mayinclude at least one position sensor and the controller may beconfigured to query the at least one position sensor to obtain the PSI,the PSI being indicative of at least one of a position and orientationof the at least one position sensor relative to a workspace. It isfurther envisioned that the controller may be further configured torender the determined guidance information as directional indications toguide the ultrasound probe to at least one of a position and orientationof the ultrasound probe for the selected view. The controller may befurther configured to display simultaneously on the rendering device twoor more of the plurality of views of the workflow. The controller may befurther configured to display on the rendering device an indication foreach of the two or more views. It is also envisioned that the controllermay be further configured to display on the rendering device the two ormore views as a single view with the two or more views anatomicallypositioned.

In accordance with embodiments of the present system, the controller maybe coupled to at least one transducer of the ultrasound probe to obtainultrasound image information using the set ultrasound probe parameters.It is further envisioned that the controller may be further configuredto reconstruct an image based upon the ultrasound image information inaccordance with the set ultrasound probe parameters. The controller maybe further configured to associate and store two or more of currentparameters, location, orientation, and ultrasound information of theultrasound probe in association with the selected view. It is alsoenvisioned that the controller may be further configured to determinewhich of the plurality of views is closest to the determined at leastone of location and orientation of the ultrasound probe and select theview based on which view is determined closest.

In accordance with embodiments of the present system, there is furtherdisclosed a method for guiding an ultrasound probe for obtainingultrasound information performed by at least one controller andcomprising acts of: determining at least one of location and orientationof the ultrasound probe based upon position sensor information (PSI);selecting a view of a plurality of views of a workflow that are storedin a memory; obtaining view setting information (VSI) for the selectedview, the VSI comprising information related to parameters and at leastone of a position and orientation of the ultrasound probe for each ofthe views; determining guidance information based upon a comparison ofthe location and orientation of the ultrasound probe and the at leastone of a position and orientation of the ultrasound probe for theselected view; and outputting the determined guidance information andsetting ultrasound probe parameters based on the parameters of the VSIfor the selected view. The method may further include an act of queryingat least one position sensor of a shape-sensing-device (SSD) to obtainthe PSI, the PSI being indicative of at least one of a position andorientation of the at least one position sensor relative to a workspace.

It is further envisioned that the act of outputting of the determinedguidance information includes acts of generating guidance instructionscorresponding to the determined guidance information and rendering thegenerated guidance instructions on a rendering device of the system. Theact of outputting of the determined guidance information may include anact of transmitting the guidance information to at least one roboticactuator to control at least one of position and orientation of theultrasound probe. It is also envisioned that the act of outputtingincludes an act of outputting the determined guidance information asdirectional indications to guide the ultrasound probe to at least one ofa position and orientation of the ultrasound probe for the selectedview. It is further envisioned that the method may include an act ofdisplaying simultaneously on the rendering device two or more of theplurality of views of the workflow.

In another aspect of the present system a user interface presentsguidance instructions and guidance information which may correspond todetermined VSI or probe parameters or choice of workflows.

In accordance with embodiments of the present system, there is furtherdisclosed a non-transitory computer readable medium that comprisescomputer instructions which, when executed by a processor, configure theprocessor to perform acts of: determining at least one of location andorientation of an ultrasound probe based upon position sensorinformation (PSI); selecting a view of at least one registered view of aworkflow; obtaining view setting information (VSI) for the selectedview, the VSI comprising information related to parameters for each ofthe registered views and at least one of a position and orientation ofthe ultrasound probe for each of the views; determining guidanceinformation based upon a comparison of the location and orientation ofthe ultrasound probe and the at least one of a position and orientationof the ultrasound probe for the selected view; and outputting thedetermined guidance information and setting ultrasound probe parametersbased on the parameters of the VSI for the selected view. It is alsoenvisioned that the instructions may configure the processor to performan act of querying at least one position sensor of ashape-sensing-device (SSD) to obtain the PSI, the PSI being indicativeof at least one of a position and orientation of the at least oneposition sensor relative to a workspace. It is further envisioned thatthe instructions may configure the processor to perform an act ofdisplaying simultaneously two or more of the plurality of views of theworkflow. The processor may further be configured by the instructions toperform the act of displaying simultaneously the two or more of theplurality of views as a single view with the two or more viewsanatomically positioned.

BRIEF DESCRIPTION OF THE DRAWINGS

The present system is explained in further detail in the followingexemplary embodiments and with reference to the figures, where identicalor similar elements are partly indicated by the same or similarreference numerals, and the features of various exemplary embodimentsbeing combinable. In the drawings:

FIG. 1 shows a perspective front view of a portion of an ultrasoundmedical navigation system operating in accordance with embodiments ofthe present system;

FIG. 2 shows a functional flow diagram performed by a process inaccordance with embodiments of the present system;

FIG. 3 shows a workflow table for each of a plurality of PTs inaccordance with embodiments of the present system;

FIG. 4A shows a portion of a screenshot of an ultrasound probe navigatedto an optimal first location corresponding with a first selected view inaccordance with embodiments of the present system;

FIG. 4B shows a portion of a screenshot of the ultrasound probe of FIG.3 navigated to an optimal second location corresponding with a secondselected view in accordance with embodiments of the present system;

FIG. 5 shows a portion of a screenshot of the ultrasound probe of FIG. 4navigated to the optimal second location corresponding with the secondselected view and including an optional location indicator such as acircle in accordance with embodiments of the present system;

FIG. 6 shows a portion of a screenshot of the ultrasound probe navigatedto an optimal location corresponding to a selected view and includingoptional location indicators such as circles in accordance withembodiments of the present system; and

FIG. 7 shows a portion of a system in accordance with embodiments of thepresent system.

DETAILED DESCRIPTION

The following are descriptions of illustrative embodiments that whentaken in conjunction with the following drawings will demonstrate theabove noted features and advantages, as well as further ones. In thefollowing description, for purposes of explanation rather thanlimitation, illustrative details are set forth such as architecture,interfaces, techniques, element attributes, etc. However, it will beapparent to those of ordinary skill in the art that other embodimentsthat depart from these details would still be understood to be withinthe scope of the appended claims. Moreover, for the purpose of clarity,detailed descriptions of well known devices, circuits, tools,techniques, and methods are omitted so as not to obscure the descriptionof the present system. It should be expressly understood that thedrawings are included for illustrative purposes and do not represent theentire scope of the present system. In the accompanying drawings, likereference numbers in different drawings may designate similar elements.The term and/or and formatives thereof should be understood to mean thatonly one or more of the recited elements may need to be suitably present(e.g., only one recited element is present, two of the recited elementsmay be present, etc., up to all of the recited elements may be present)in a system in accordance with the claims recitation and in accordancewith one or more embodiments of the present system.

For the sake of clarity, embodiments of the present system will be shownand described with respect to shape sensing devices (SSDs) such asshape-sensing fibers as may be employed using FORS methods and the like.However, it is also envisioned that embodiments of the present systemmay be compatible with other tracking systems which may sample multipledata points sequentially or simultaneously such as EM tracking methodsand the like. Further, it should be assumed that SSDs of the presentsystem may be used alone or with a sheath such as a catheter, aguidewire, a surgical tool, an imaging tool (e.g., an ultrasound probe),and the like.

FIG. 1 shows a perspective front view of a portion of an ultrasoundmedical navigation system 100 (hereinafter system 100 for the sake ofclarity) operating in accordance with embodiments of the present system.The system 100 may include one or more of an SSD 102, an ultrasoundprobe 104, a base 108, a support platform 110, a controller 122, arobotic controller 140, a network 120, a memory 124, sensors 132, and auser interface (UI) 126. The controller 122 may control the overalloperation of the system 100 and may communicate with one or more of theSSD 102, the ultrasound probe 104, the base 108, the support platform110, the memory 124, the sensors 132, the robotic controller 140, andthe UI 126 using any suitable wired and/or wireless communicationmethods. For example, the controller 122 may communicate with theultrasound probe 104 via the network 120. The robotic controller 140 maybe operative to robotically manipulate one or more surgical implementsof the present system such as the ultrasound probe 104 to a desiredposition and/or orientation. Accordingly, the robotic controller 140 mayinclude one or more actuators, arms, etc. to manipulate the ultrasoundprobe 104 along or about one or more axes (e.g., multiple axes such as7, etc.).

The network 120 may include any suitable communication link such as awide-area network (WAN), a local-area network (LAN), the Internet, asystem bus, a proprietary bus, the Internet, an intranet, a wired bus, awireless bus, etc. Accordingly, a user may communicate with the systemusing local and/or remote communication methods. The memory 124 mayinclude any suitable non-volatile memory in which information such asoperating instructions, information generated by the system, user inputsand/or settings, historical information, operating settings and/orparameters, identification information, user information, patientinformation, etc., may be stored.

The sensors 132 may include sensors which may obtain correspondingsensor information and provide this sensor information to the controller122 for further processing. The controller 122 may query one or more ofthe sensors 132 for the sensor information. For example, the sensors 132may include optical-shape-sensors which may sense a shape of the SSD 102and provide this information to the controller 122 which may determineposition and/or orientation of one or more portions of the SSD 102 inaccordance with embodiments of the present system. The sensors 132 maybe distributed throughout the system 100 and may further include sensorssuch as touch sensors, a keyboard, etc., with which a user may enterinformation into the system. The sensors 132 may further include an EMtracking sensor. Further, the sensors 132 may include position sensorswhich may provide position/orientation information related to aposition/orientation of the support platform 110, the patient 101, thebase 108, an EM tracker, etc.

The UI 126 may include any suitable user interface which may renderinformation for the convenience of user such as graphical userinterfaces (GUIs) generated by the system and/or image information.Accordingly, the UI 126 may include a speaker (SPK), a display 128(e.g., a touch-screen display, etc.), haptic device (e.g., vibrators,etc.). The support platform 110 may be any suitable support platformwhich may support an object such as a patient 101 in a desired positionand/or orientation for a procedure such as an interventional procedure.The support platform 110 may include actuators which may move thesupport platform under the control of the controller 122.

The base 108 may include any suitable device base or bases (physical orvirtual) which may function as a launch fixture for one or more of theultrasound probe 104 and the SSD 102. Accordingly, a position and/ororientation at least a portion of one or more of the ultrasound probe104 and the SSD 102 may be determined relative to a reference frame. Thebase 108 may include a plurality of bases. The reference frame mayinclude any suitable reference frames such as a reference frame of thesupport platform 110, a reference frame of the patient 101, a commonreference frame, etc. However, for the sake of clarity, it will beassumed that the reference frame may refer to the reference frame of thepatient 101. Thus, the reference frame may correspond with the anatomyof the patient 101. Accordingly, the system 100 may employ a workspaceas defined by the patient 101 for the sake of clarity. This workspacemay be referred to as the patient workspace. The base 108 may be coupledto any suitable fixture such as a C arm, the support platform 110, etc.

The ultrasound probe 104 may include any suitable ultrasound probe for aprocedure being performed. For example, in the present embodiments theultrasound probe 104 may be assumed to include a transesophageal echo(TEE) probe which may obtain ultrasound image information forconstructing images of the heart of the patient 101 for one or moreviews (e.g., viewpoints). Further, it will be assumed that for eachview, position, orientation, settings and/or parameters of theultrasound probe 104 may be varied. The ultrasound probe 104 may includean ultrasound sensor (e.g., transducer) array 106 for capturing, theultrasound image information for each view. The ultrasound sensor array106 may be situated at, or adjacent to, a distal end 107 of theultrasound probe 104. The ultrasound probe 104 may include an openingsuitable for receiving the SSD 102 so that a position and/or orientationof, for example, a portion of the ultrasound probe 104 such as theultrasound sensor array 106 may be easily determined. However, andwithout limitation, it should be understood that the opening may includea notch or tab suitable for receiving a portion of the SSD 102 as may bedesired. This notch or tab may be located at a desired position on theultrasound probe 104 such as at or near the distal end 107 of theultrasound probe 104. The opening may be configured to receive the SSD102 such that the position and orientation of the SSD 102 relative tothe opening may be defined. For example, the opening may be keyed toreceive the SSD 102 only in a single position such that position and/ororientation of the SSD 102 may be determined relative to the ultrasoundprobe 104. The position and/or orientation may be determined using aposition and/or orientation vector. However, in yet other embodiments,the ultrasound probe 104 may be set at a known distance from the SSD 102(e.g., see, FIGS. 3-6).

The ultrasound probe 104 may be located at a distal end of a sheath 105.The sheath 105 may include one or more openings along a length thereofthrough which the SSD 102 may be inserted to reach the ultrasound probe104. Similarly, the ultrasound probe 104 may be inserted through thesame or another opening. Accordingly, the ultrasound probe 104 and theSSD 102 may be simultaneously located within the sheath 105. However, inaccordance with yet other embodiments, the ultrasound probe 104 and theSSD 102 may be inserted through different lumens.

The ultrasound probe 104 may communicate with the controller 122 usingany suitable wired and/or wireless communication method. For example,the ultrasound probe 104 may communicate with the controller 122 via thenetwork 120. The controller 122 may control the ultrasound sensor array106 to generate ultrasound information suitable for generating anultrasound image of a desired view (e.g., a desired viewpoint). Theultrasound probe 104 may include identifying landmarks which may berecognized in one or more images of the system such as x-ray images anda corresponding position and/or orientation of the ultrasound probe 104may be determined. Thereafter, a registration may be performed toregister images obtained from the ultrasound probe 104 to the x-rayimages, etc. and a position and/or orientation of the ultrasound probe104 may be determined based upon the x-ray images of the ultrasoundprobe 104. Suitable registration methods may be discussed in U.S. PatentPublication No. 2012/0245458 which shows registration between X-ray andultrasound and International Patent Publication Nos. WO 2014/053934A1which shows registration between FORS and ultrasound WO2015010859 whichshows registration between FORS and X-ray, the contents of each of whichare incorporated herein by reference. As readily appreciated, othersuitable image registration methods may also be suitably applied forregistering portions of the present system.

The SSD 102 may include at least one sensor which may provide SSDinformation suitable for determining a position and/or orientation of atleast a portion of the SSD 102 relative to a desired reference framesuch as a reference frame of the patient 101 (e.g., the patientworkspace). For the sake of clarity, it may be assumed that the SSD 102may pass through at least a portion of the opening of the ultrasoundprobe 104 such that a distal end 103 of the SSD 102 may be situated atthe ultrasound probe 104. For example, the SSD 102 may be coupled to theultrasound probe. However, in yet other embodiments, the SSD 102 may besituated a known distance from the ultrasound probe 104, if desired, anda known offset distance and/or orientation may be used to determine theposition and/or orientation of the ultrasound probe 104. The SSD 102 mayfurther pass through or within a known path such as a known path 111(which may be coupled at a known position and/or orientation to, forexample, the base 108) which may be recognized by the system 100 forregistration. The ultrasound probe 104 may include a known path, forexample so that when situated within this known path, this known pathmay be recognized (e.g., through an analysis of SSD information), and aposition and/or orientation of ultrasound probe 104 or portions thereofsuch as portions at or adjacent to this known shape, may be determined.

Referring back to the controller 122, the controller 122 may control theoverall operation of the system 100 and may include one or more logicdevices such as processors 130 (e.g., microprocessors (μP), etc.) havingmultiple interconnected semiconductor devices such as transistors,gates, impedance devices, metallization connections and the like,discrete and/or distributed logic gates and switching devices, and/orthe like. The controller 122 may include an interrogation module 134and/or a registration module 136 which may include hardware, softwareand/or firmware devices with instructions stored in a memory thereofand/or the memory 124, which when executed by the processor cause theprocessor to perform one or more desired functions.

The interrogation module 134 may be operative to obtain information fromthe SSD 102 (e.g., via an interrogation process) such as SSD information(SSDI) (which will be described below) and which may indicate a pathtravelled by the SSD 102 over time and/or a shape of one or moreportions of the SSD 102. The path may be determined serially (e.g., overtime) and/or simultaneously (e.g., at a single time). The SSDI may thenbe processed to determine a location and/or orientation of theultrasound probe 104. When placed through or within a known path or oneor more known paths (e.g., 111), The SSD may assume a shape of thecorresponding known more known paths and form corresponding SSDI. Thecontroller 122 may recognize this known path (e.g., through an analysisof the SSDI) and determine a position and/or orientation of one or moreportions of the SSD 102 relative to the correspondingly recognized knownpath. However, it is also envisioned that a position and/or orientationof a known path (e.g., a known path in the ultrasound probe 104) may bedetermined based upon the SSDI.

The registration module 136 may be operative to register the positionand/or orientation of the SSD 102 and/or the ultrasound probe 104relative to one or more reference coordinate systems such a workspace ofthe patient 101 (e.g., the patient workspace which may reflect ananatomy of the patient 101) as used in the current embodiments. Forexample, the registration module 136 may register one or more portionsof the system (and/or information obtained therefrom) such as an EMtracking system (e.g., an EM generator and/or an EM sensor), an x-rayimager, the SSD, and the ultrasound probe 104 for example to a workspaceof the patient 101. The registration module 136 may further register oneor more portions of the system 100 such as the base 108, known shapes(e.g., the shape of known path 111), etc., as may be desired, to a knownworkspace such as the patient workspace. For example, it is envisionedthat the registration module 136 may register images obtained from oneor more imaging modalities of the system such as an X-ray images (e.g.,which may be obtained in real time), ultrasound images (e.g., anultrasound views), patient anatomy, a position of the SSD 102 (e.g., viaSSD information), one or more workspaces, etc. For example, the X-rayimages may be registered to an ultrasound view, and/or an SSD, which mayultimately be registered to a workspace of the patient 101 (e.g., thepatient anatomy for the sake of clarity) or vice versa. An order ofregistration may vary. The system 100 may include software to performregistration such as an echo navigator, etc.

The SSD 102 may extend from the base 108 for a given length (L_(ssd))and may provide signals such as SSDI indicative of its position and/ororientation along at least a portion of its length (L_(ssd)). The SSDImay further include information related to a shape of the SSD 102 at oneor more locations thereof, if desired. The SSD 102 may be formed usingany suitable shape-sensing device such as a Fiber Optic RealShape™(FORS) fiber or the like which may provide sensor information (e.g.,SSDI) from a plurality of sensors indicative of position and/ororientation of a plurality of locations along its length L_(ssd). Eachof the sensors may provide information related to, for example, aposition (e.g., an x, y, z coordinate, etc.) and/or orientation (e.g.,twist of a corresponding fiber) of the corresponding sensor. Theplurality of shape sensing locations may approach a continuum oflocations, as desired. However, generally, the plurality of shapesensing locations may be set apart from each other by a desired distancesuch as 40 μm or other suitable distance. Suitable SSDs 102 may include,for example, a shape sensing fiber (SSF), an EM-based tracking devicewith at least one EM sensor such as an EM sensor located at a tipthereof, etc., and/or combinations thereof, such as described in U.S.Patent Publication No. 2013/0317356 to Ramachandran et al., which isincorporated herein by reference in its entirety. In accordance withembodiments of the present system, the position and/or orientationsensors may be active light emitting diodes, passive reflectors such asspheres, optical and/or EM coils, and/or radioactive or radiopaquemarkers which are identifiable based on imaging such as X-ray and/ornuclear based imaging. Similarly to the optical sensors, the EM sensorsand/or other sensors/markers may be situated at one or more locationsapart from each other as may be desired, or a shape may be reconstructedfrom a single point sensor by accumulating a history of positions as theSSD is passed through a path.

During operation, the SSDI may be obtained by interrogating the SSD 102.The interrogation may employ optical and/or EM interrogation techniqueswhich may correspond with a type of the SSD employed by the system. Forexample, the SSF may employ Fiber Optic RealShape™ (FORS) interrogationtechniques to determine position and/or orientation thereof, while theEM-based tracking device may employ EM interrogation methods to obtainthe SSDI. However, it should be understood that these interrogationtechniques may or may not be exclusive of each other. For example, twoor more of these techniques may be utilized together. Further, theoptical interrogation technique may interrogate at least one opticalsensor of the SSD synchronously in time and the EM interrogationtechnique may interrogate at least one EM sensor of the SSD sequentiallyin time and/or vice versa.

Regardless of the type of interrogation technique, the interrogation mayobtain the SSDI which may then be processed to perform a registration ofthe SSD 102 to a reference workspace coordinate system (e.g., thepatient workspace) as described in this application and/or to determinea location and/or orientation of one or more portions of the SSD 102such as a distal end 103 of the SSD 102. The controller 122 may becommunicatively coupled (using any suitable method such aselectronically, optically, etc.) to the SSD 102 so that the SSD 102 maybe interrogated. For the sake of clarity, it is assumed that theworkspace of the patient 101 may be referred to as a referenceworkspace. However, it should be understood that there may be severalreference workspaces in a system operating in accordance withembodiments of the present system. In accordance with embodiments of thepresent system, the controller 122 may drive an EM field generator whichmay obtain EM field information from the SSD 102 when the employing anEM-type SSD.

FIG. 2 shows a functional flow diagram performed by a process 200 inaccordance with embodiments of the present system. The process 200 maybe performed using one or more processors, computers, controllers, etc.,communicating over a network and may obtain information from, and/orstore information to one or more memories which may be local and/orremote from each other. The process 200 may include one of more of thefollowing acts. In accordance with embodiments of the present system,the acts of process 200 may be performed using one or more suitablecoordinate registration systems operating in accordance with embodimentsof the present system. Further, one or more of these acts may becombined and/or separated into sub-acts, as desired. Further, one ormore of these acts may be skipped depending upon settings. For the sakeof clarity, the process may be described with reference to a singleultrasound probe. However, without limitation, it should be understoodthat the process may employ a plurality of ultrasound probes each ofwhich may be include a separate workflow such as a sub-workflow. Inoperation, the process may start during act 201 and then proceed to act203. Further, it will be assumed that the ultrasound probe may include aknown path.

During act 203, the process may obtain a workflow for a currentprocedure. This workflow may be referred to as a current workflow. Theworkflow may be obtained from workflow information (WI) which mayinclude information corresponding with a current procedure or workflowif known. The workflow may have associated with it a plurality ofregistered views (e.g., viewpoints) each of which may image informationand corresponding location (e.g., x, y, z, etc.), orientationinformation (e.g., α, β, γ, etc.), and/or ultrasound parameter settings.In accordance with embodiments of the present system, these views may bestored as a view library for each workflow. The workflow may correspondwith a procedure-type (PT) being performed and may be obtained from amemory of the system in accordance with the PT being performed and/ormay be designated by a user. Thus, a workflow may be defined inaccordance with a procedure-type (PT) and/or a user (performing theprocedure). For example, in accordance with embodiments of the presentsystem, a workflow for a PT may include one or more procedures (e.g.,sub-procedures) which may be performed in any desired order as may beset by the system and/or the user. For example, FIG. 3 shows a workflowtable 300 for each of a plurality of PTs 301 x in accordance withembodiments of the present system. With reference to FIG. 3, assumingthat there are M procedure types 301A through 301M, wherein M is aninteger (generally 301 x), each procedure type (e.g., appendectomy,cardio ablation, etc.) may have a corresponding workflow 303A through303M (e.g., workflow 1 through M, where M is an integer) associatedtherewith. Each workflow may have corresponding information associatedtherewith such as one or more of registered views (RVs) 303 andassociated information such as coordinate information (CI), orientationinformation (01), view order information 307 (e.g., includinginformation related to suitable orders to obtain views as desired), andparameter information (e.g., for setting parameters of the ultrasoundprobe) for each registered view. For example, with reference toregistered view 303-1, there may be corresponding view settinginformation (VSI) which may include information such as coordinateinformation (CI) (x, y, z), orientation information (OI) (α, β, γ), andparameter information (Param). The parameter information may includeinformation related to parameters and/or other settings for anultrasound probe used by the present system to obtain ultrasound images.The workflow table 300 may be stored in a memory of the system and maybe set/reset by the system and/or the user. For example, the system mayrender the workflow table 300 so that a user may select a PT and see acorresponding workflow. Further, each workflow may be registered to aworkspace of the system. Further, each registered view may include oneor more landmarks as may be desired.

It is further envisioned, that the plurality of registered views may bestored in a memory of the system and may be selected based upon a typeof procedure being performed. For example, a procedure of type I mayhave a workflow which may include 5 views assigned to it, while aprocedure of type II (different from type I) may have a workflow whichmay include 6 views assigned to it. For example, in accordance withembodiments of the present system a Left Ventricular Obstruction Track(LVOT) setting may include a workflow including predefined registeredviews for determining, for example, a left ventricular obstructiontrack. Other procedures may correspond with, for example, a transseptalpuncture, a mitral valve navigation, deploying an LAA closure, etc.Thus, a type of procedure may be determined during an initial setup, forexample automatically determined by a processor based on stored patientdata and/or selected by a user, etc., and the view or views may beselected based upon the determined procedure. Further, the views mayfurther be defined by a user and/or procedure type. Thus, user A mayhave different views stored than user B regardless of the proceduretype. After completing act 203, the process may continue to act 205.

During act 205, the system may obtain position sensor information (PSI)which may be indicative of a position and/or orientation of at least aportion of the SSD such as a distal end of the SSD. The PSI may includeSSD information (SSDI) which may be obtained by interrogating the SSD toobtain the SSDI from a plurality of shape-sensing sensors each at acorresponding location of the SSD which sensor information may beindicative of position and/or orientation of a plurality ofcorresponding locations along the length L_(ssd) of the SSD. Forexample, the system may employ a FORS method to interrogate the SSD toobtain the SSDI. Further, the plurality of shape-sensing locations mayapproach a continuum of locations, as desired. Thus, during aninterrogation, the system (e.g., a suitably programmed processor of thesystem) may interrogate the at least one sensor of the SSD sequentiallyand/or synchronously over time to obtain position information from atleast one sensor and form corresponding SSDI. This SSDI may then bereconstructed by the processor to determine a path (P) of the SSD and todetermine a position and/or orientation of at least a portion of the SSDsuch as a distal end of the SSD.

Suitable SSDs may include, for example, a shape sensing fiber (SSF), anEM-based tracking device with at least one EM sensor such as an EMsensor located at a tip thereof, etc., and/or combinations thereof, suchas described in U.S. Patent Application Publication No. 2013/0317356 toRamachandran et al. The position and/or orientation sensors may includeactive light emitting diodes, passive reflectors such as spheres,optical and/or EM coils, and/or radioactive or radiopaque markers whichare identifiable based on imaging such as X-ray and/or nuclear basedimaging. Similarly to the optical sensors, the EM sensors and/or othersensors/markers may be situated at one or more locations apart from eachother as may be desired, or a shape may be reconstructed from a singlepoint sensor by accumulating a history of positions as the SSD is passedthrough a known path. Accordingly, during this act an interrogationmodule (e.g., in a form of one or more hardware devices, softwareportion programming a portion of the processor and/or a dedicatedprocessor) may be operative to obtain information from the SSD (e.g.,via an interrogation process) such as the SSDI and which may indicate apath travelled by the SSD 102 over time and/or a shape of one or moreportions of the SSD 102. After completing act 205, the system maycontinue to act 207.

During act 207, the system may determine at least one of position andorientation of the ultrasound probe (or a particular ultrasound probe ifa plurality of ultrasound probes is desired) based upon the positionsensor information (PSI). The PSI may be indicative of at least one of aposition and orientation of the at least one position sensor of the SSDrelative to the workspace. As the SSD may be positioned in a knownrelation (e.g., a known offset) to the ultrasound probe, the positionand/or orientation of the ultrasound probe may be determined inaccordance with the PSI and/or an SSD-to-ultrasound probe transformationfor example assuming a known offset. This known offset may be previouslydetermined and stored in a memory of the system for later use and/or theoffset may be determined or confirmed during use.

For the sake of clarity, the workspace may be defined as a workspace ofthe system and may be a general workspace or may be workspace definedrelative to an object being scanned such as a patient. Thus, for thesake of clarity, the workspace may be defined as a workspace of thepatient and, thus, may correspond with an anatomy of a patient such asthe patient and/or a common workspace (e.g., common to the devices ofthe present system). Further, it will be assumed that a registrationprocess to register one or more of an X-ray imager, the SSD, and theultrasound probe to a workspace of the system may have been performedand/or may be performed during use.

It is further envisioned that the system may employ image analysis(e.g., of a captured ultrasound image) to determine a location of theultrasound probe relative to a known view such as a registered viewwhich may be registered in a memory of the system. Accordingly, thesystem may obtain ultrasound information from the ultrasound probe anddetermine a corresponding position and/or orientation of the ultrasoundprobe based upon this ultrasound information using any suitable methodsuch as using image analysis of the ultrasound information and thereby,form corresponding PSI relative to the known anatomy of the patient and,thus, the patient workspace. Thereafter, the process may update the PSIaccordingly to include this information. For example, the ultrasoundimage information may include one or more landmarks which may correspondwith known landmarks in a registered view(s) relative to the workspace.Accordingly, difference information between location and/or orientationof the known landmarks and corresponding information in the ultrasoundimage information may be determined and thereafter this differenceinformation may be used to determine a position and/or orientation ofthe ultrasound probe relative to the workspace.

In addition, position information and/or orientation for interventionaldevices such as one or more additional ultrasound devices, catheters,ablative devices, etc., may be acquired for example using a SSD or otherdevice/system for acquiring such information such as described herein.Further, one or more images may be acquired of the interventionaldevices which may be saved, recalled and rendered. For example, the oneor more images of the interventional device(s) may be rendered in animage including the ultrasound instrument, a view of a workflow, etc.,such as illustratively depicted in FIGS. 4A, 4B, 5 and 6. Aftercompleting act 207, the system may continue to act 209.

During act 209, the system may select a registered view (hereinafter theselected view) from a plurality of registered views of the currentworkflow using any suitable method. For example, in accordance withembodiments of the present system, the selected view may be selectedfrom the plurality of registered views based upon the view orderinformation (e.g. view order 1, 3, 4, 5, 7, etc.), a user selection(e.g., select registered view 3, etc., which may be selected using auser input device of the system such as a touch-screen, a keyboard, amicrophone, etc.). However, it is also envisioned that the order of theplurality of registered views, may be selected from a default selection.Thus, if there are five consecutive registered views previously definedfor the current workflow, the system may obtain these registered viewsin any suitable order (e.g., first to last, vice versa, randomlyselected, based upon location of the ultrasound probe (nearest tofurthest), etc.) as may be set by the system and/or user.

In accordance with embodiments of the present system, the selected viewmay be selected based upon the determined location and orientation of anultrasound probe. For example, if is it determined that the determinedposition of the ultrasound probe is within a threshold distance Δ_(DIS)of a registered view (e.g., a view of the views), the system may setthis view as a selected view. Similarly, in a case where it isdetermined that the determined orientation of the ultrasound probe iswithin a threshold orientation Δ_(ORIENT) of a view (e.g., a given viewof the views), the system may set this view as a selected view.

In accordance with embodiments of the present system, it is envisionedthat the system may select a view from the plurality of views when boththe determined position of the ultrasound probe is within a thresholddistance Δ_(DIS) of a view and the determined orientation of theultrasound probe is within a threshold orientation Δ_(ORIENT) of theview. The system may define the position and/or orientation of theultrasound probe using a vector or the like. In accordance withembodiments of the present system each view may have defined thresholdssuch as a threshold orientation Δ_(ORIENT) (e.g., defined orientation+/−Δ_(ORIENT)) and a threshold distance Δ_(DIST) (e.g., defined position+/−Δ_(DIST)) defined.

In accordance with other embodiments, the view may be selected by a usersuch as a clinician performing an ultrasound-based imaging procedure.Accordingly, the process may provide one or more keys (e.g., hard, soft,etc.), menu items, etc., that the user may choose to select (e.g., bytoggling, etc.) a registered view of a plurality of registered views.For example, the system may render one or more menu items (e.g.,circles) on an image which may represent a view. The user may thenselect any of these menu items to select the corresponding registeredview and the system may set this view as a selected view. These menuitems may be superimposed upon an image of a region-of-interest (ROI).Thus, with reference to FIG. 3, the process may render registered viewsfor the present workflow and a user may select one of these registeredviews, by, for example, touching the corresponding registered view.After completing act 209, the system may continue to act 211.

During act 211, the system may obtain view setting information (VSI) forthe selected view. The VSI may include information related to each ofthe registered views. For example, the VSI may include information suchas location and/or orientation of a view and/or of an ultrasound probeto obtain the view (e.g., to image the view) as well as parametersand/or settings (e.g., hereinafter both of which may be referred to asparameters for the sake of clarity) for the ultrasound probe for eachview such as one or more of focus, depth, scan angle, etc. Theseparameters may be stored in association with each of the registeredviews. The VSI may be stored in a memory of the system. After completingact 211, the process may continue to act 213.

During act 213, the system may determine guidance information. Theguidance information may be determined by the system and may provideguidance to move (e.g., linearly, non-linearly, and/or rotationally) theultrasound probe from a current position to a position which correspondswith the selected view. Thus, for example, the system may determinedifference information between a current position and/or orientation ofthe ultrasound probe and the desired position and/or orientation,respectively, of the ultrasound probe. The desired location and/ororientation of the ultrasound probe may be defined as a position and/ororientation which corresponds with that of the selected view. Thus, theguidance information may for example be based upon differenceinformation (e.g., a difference between a desired position and/ororientation for a view and a current position and/or orientation of theultrasound probe. However, any suitable method to determine the guidanceinformation may be employed. After completing act 213, the system maycontinue to act 215.

During act 215, the system may output the determined guidanceinformation. For example, in accordance with embodiments of the presentsystem the system may form guidance instructions which may correspondwith the determined guidance information and generate instructions, suchas through use of a graphical user interface (GUI) including theseguidance instructions in accordance with the determined guidanceinformation. The system may then render this GUI on a rendering deviceof the system such as on a display, a speaker, etc. The guidanceinstructions may be based upon a rendering method employed by thesystem. For example, in accordance with some embodiments, the system maygenerate instructions such as move forward, reverse, turn right, turnleft, etc., and/or graphic representations thereof, and display theseinstructions on a display of the system.

For example, in accordance with yet other embodiments, the system mayrender information related to the determined guidance information usingany suitable guidance method such as lighting hard or soft keys,light-emitting-diodes (LEDs), driving a haptic device (e.g. provided ona handle of the ultrasound probe, etc.), etc. It is further envisionedthat the system may determine an orientation of the user (e.g., theultrasound clinician) and convert the guidance information accordinglyso that the user may move the ultrasound probe to the desired positionand/or orientation by manipulating controls of the ultrasound probe inaccordance with guidance information rendered by the system.

It is also envisioned that the system may employ a robotic controller(e.g., a robotic actuator or the like) to robotically manipulate theultrasound probe to a desired position and/or orientation. For example,the system may provide the guidance information to a robotic controllerof the ultrasound probe which may then process the guidance informationand provide corresponding information to one or more actuators of anultrasound probe to effect a corresponding movement of the ultrasoundprobe. For example, the actuators may provide linear and/or rotationalmovement of the ultrasound probe such that the ultrasound probe may bemoved to the determined position and/or orientation as set forth by theguidance information such that the ultrasound probe may be manipulatedto a desired position and/or orientation which corresponds with aposition and/or orientation of the selected area. After completing act215, the system may continue to act 217.

During act 217, the system may determine whether the ultrasound probe isat, or is substantially at a position and/or orientation of the selectedview (e.g., within a final threshold distance Δ_(D) and/or a finalthreshold orientation Δ_(R) of the selected view). Accordingly, in acase where it is determined that the ultrasound probe is at, or issubstantially at (e.g., within a final threshold position Δ_(D) and/or afinal threshold orientation Δ_(R)) a location and/or orientation of theselected view, the process may continue to act 219. However, in a casewhere it is determined that the ultrasound probe is not at, or notsubstantially at (e.g., not within a final threshold position Δ_(D)and/or a final threshold orientation Δ_(R)) a position and/ororientation of the selected view, the system may repeat act 213. Todetermine whether the ultrasound probe is at, or is substantially at aposition and/or orientation of the selected view, the process may obtaininformation on a current position and/or orientation of the ultrasoundprobe, update the difference information and compare this updateddifference information with the final threshold position Δ_(D) and/or afinal threshold orientation Δ_(R), respectively. Accordingly, in a casewhere corresponding portions of the updated difference information aregreater than the final threshold position Δ_(D) and/or the finalthreshold orientation Δ_(R), the system may determine that theultrasound probe is not at, or not substantially at (e.g., not within afinal threshold position Δ_(D) and/or a final threshold orientationΔ_(R)) a position and/or orientation of the selected view. However, in acase where corresponding portions of the updated difference informationare less than or equal to (e.g., not greater than) the final thresholdposition Δ_(D) and/or the final threshold orientation Δ_(R), the systemmay determine that the ultrasound probe is at, or substantially at(e.g., within a final threshold position Δ_(D) and/or a final thresholdorientation Δ_(R)) a position and/or orientation of the selected viewand thereby, the system may continue to act 219.

During act 219, the system may indicate that the ultrasound probe is ata desired position (e.g., is at, or is substantially at a positionand/or orientation) of the selected view. In other words, the ultrasoundprobe is positioned for the selected view. Accordingly, the system mayrender information indicating such on a rendering device of the system.For example, the system may highlight an overlay of the ultrasound probeusing a green highlight to indicate to a clinician that it is in thedesired position. The system may further provide information indicatingsuch to one or more applications of the system such as to the roboticcontroller to stop motion and/or to lock the ultrasound probe in thecurrent position and/or orientation (e.g., to prevent undesirablemovement) and/or provide such information to an ultrasound image captureapplication which may use this information to begin an ultrasound imagecapture process as will be described below with respect to act 221. Inyet other embodiments, the system may apply a brake and/or provideinstructions to the clinician to apply a brake to prevent inadvertentmovement of the ultrasound probe. After completing act 219, the systemmay continue to act 221.

During act 221, the system may capture ultrasound information at alocation corresponding to the selected view and/or with the parameterscorresponding to the selected view. This captured ultrasound informationmay then be processed to generate ultrasound image information suitablefor rendering on a display of the system. After completing act 221, thesystem may continue to act 223.

During act 223, the system may store the captured ultrasound informationin any suitable format (e.g., raw and/or processed) in a memory of thesystem in association with the selected view. Accordingly, the systemmay recall this ultrasound information at a later time in associationwith the selected view as desired. The ultrasound information may bestored as still and/or video image information in association with theparameters and/or in association with the corresponding selected view.Parameters including adjustments to parameters may be similarly storedin association with the corresponding selected view. In this way,historical information may be obtained including which views areaccessed and/or adjusted. The historical information may be utilized todetermine which views and/or parameters are used to produce a set ofdefault views and corresponding parameters for a given workflow. Forexample, if for a given workflow, parameters are adjusted more than halfthe time from the parameters stored as a default, an average of theadjustments may be utilized for adjusting the parameters to a newdefault. Naturally views and/or parameters for a given patient relatedto one or more of the views may also be saved and recalled as desired.After completing act 223, the process may continue to act 225.

During act 225, the process may determine whether there are any otherviews in the current workflow to capture. Accordingly, in a case whereit is determined that there is another view to capture, the process mayrepeat act 209 for the next view. However, in a case where it isdetermined that there are no other views to capture, the process maycontinue to act 227 where the process may end.

It is further envisioned that the system may store ultrasound imageinformation, corresponding position, orientation and/or parameterinformation for the ultrasound probe in association with a view upon arequest from a user. The system may then generate and render a menuindicating the views. The user may then select a view and the system mayrender the captured view and/or guidance information (from a currentultrasound location) to the desired view and/or parameter informationrelated to the view as may be desired by the user and/or as may be setby the system (e.g., a default setting may determine what the systemrenders).

In accordance with embodiments of the present system, a user may toggleto select/deselect registered views. Moreover, a learning process maylearn a procedure as it is performed by a user and this process maydetermine the views, parameters, etc.

FIG. 4A shows a portion of a screenshot 400A, such as may be provided bya display, of an ultrasound probe 404 such as a FORS TEE situated at anend of a catheter 405. The ultrasound probe 404 may be positioned tocapture a desired view such as an image volume 407 and may be navigatedto an optimal first location corresponding with a first selected view inaccordance with embodiments of the present system. The patient (notshown in the figure for clarity) may be positioned on a supportstructure 410, such as a surgical table. The system may determine aposition of the ultrasound probe 404 using any suitable method such asby tracking a FORS SSD using a FORS tracking method or the like asdescribed herein. Element 402 may represent a surgical instrument, suchas a catheter, ablation instrument, etc. that is present in the view.Guidance messages such as arrows 480 may be rendered for guiding a userto control a location and/or orientation of the ultrasound probe 404 toa desired view location and/or orientation. The arrows 480 may behighlighted in solid to indicate a desired motion. Further, the arrows480 may be highlighted using green for example when it is determinedthat the ultrasound probe 404 has reached a desired location. Further,in a case wherein more than one ultrasound probe is employed, guidanceinformation for one or more (e.g., each) of the ultrasound probes may bedetermined and rendered in association with the corresponding ultrasoundprobe. FIG. 4B shows a portion of a screenshot 400B of the ultrasoundprobe 404 of FIG. 4A navigated to an optimal second locationcorresponding with a second selected view in accordance with embodimentsof the present system. An SSD as described herein may be interrogated todetermine the location of the ultrasound probe 404.

FIG. 5 shows a portion of a screenshot 500 of the ultrasound probe 404of FIG. 4A navigated to the optimal second location corresponding withthe second selected view and including an optional location indicatorsuch as a circle 590 in accordance with embodiments of the presentsystem. When the ultrasound probe 404 is determined to be at, orsubstantially at, the corresponding view (e.g., the second view in thecurrent embodiments), the system may render an indicator such as thecircle 590 indicating that the ultrasound probe 404 is at thecorresponding location. However, in yet other embodiments, the circle590 may include a highlight (e.g., green to indicate that the ultrasoundprobe 404 is at the proper location, red to indicate not at the properlocation corresponding to the selected area, etc.). Further, the colorsmay also indicate the ultrasound parameter settings such as depth, focusetc. In yet other embodiments, it is envisioned that the system maygenerate arrows indicative of a direction and/or orientation to move theultrasound probe to reach the desired area (location) of a correspondingview so as to capture the image volume 407. An SSD as described hereinmay be interrogated to determine the location of the ultrasound probe404.

FIG. 6 shows a portion of a screenshot 600 of the ultrasound probe 404navigated to an optimal location corresponding to a selected view andincluding optional location indicators such as circles 690 in accordancewith embodiments of the present system. The circles 690 may indicateoptimal locations for different views and may be colored or otherwisehighlighted to indicate ultrasound parameter settings for each of thesedifferent views. For example, the colors may indicate the ultrasoundparameter settings such as depth, focus etc. The different views may beeither from a previously navigated position or built up from a model ordatabase. An SSD may be interrogated to determine the location of theultrasound probe 404 so that the ultrasound probe 404 may capture thecorresponding view.

Accordingly, embodiments of the present system may provide a method inwhich FORS methods may be combined with ultrasound imaging to recordand/or remember an ultrasound probe location and correspondingultrasound parameter information for a location (e.g., a position and/ororientation of the ultrasonic probe) for a view. For example, aclinician such as an echocardiographer may navigate to a desiredlocation and adjust the ultrasound parameters such as focus, depth, scanangle etc. The system may then store these settings in association withthe desired location (e.g., a view) for later use. It is furtherenvisioned that the embodiments of the present system may provide a useran option to select any setting that is desired to be stored for lateruse in association with a selected view.

It is further envisioned that embodiments of the present system mayautomatically detect when an optimal position and/or view has beenobtained based upon a comparison of optional positions and/or viewsstored in a memory of the system. When an optimal position is determinedto be obtained, the system may inform a user of the same and may storesettings and/or parameters used by an ultrasound probe to obtainultrasound information of a corresponding view. It is further envisionedthat a user may define how many positions and/or orientations (e.g.,each corresponding to a view) are to be stored in a memory of the systemdepending on anatomy of a patient and/or a procedure being performed.

After the views and associated information are stored in a memory of thesystem, when a user (e.g., a clinician, a physician, etc.) desires toreturn to a previously-saved view (e.g., which may be considered aregistered view), such as an LVOT, the system may provide a guidancefunction which may render appropriate guidance information (e.g.,graphics such a arrows, etc., audio information (go right, etc.), hapticfeedback, etc.) for a user to guide an ultrasound probe back to thepreviously saved view. In addition, ultrasound information (e.g.,images) may then be captured for viewing using associated parametersthat were previously stored in association with the corresponding savedview and/or may be recalled from a memory. It is further envisioned thata user may select a previously saved view, and the system may control aclosed-loop robotic controller to automatically move an ultrasound probeto a desired position and set parameters, such as ultrasound parameters,in accordance with corresponding parameters so as to generate an optimalimage.

It is also envisioned that embodiments of the present system may storeoptimal positions and corresponding ultrasound settings which may beused to generate optimal views for different procedures and differentparts of these different procedures such as performing a transseptalpuncture, navigating to the mitral valve, deploying a left atrialappendage (LAA) closure device, mitral clip, etc. During theseprocedures, the system may provide guidance so that workflow of acorresponding procedure may be simplified and shortened. This may beespecially so when performing procedures in difficult anatomies and/orwhen a good image may be difficult to obtain.

It is further envisioned that embodiments of the present system maysupport a combination of ultrasound probes (e.g., two or more ultrasoundprobes) including operating parameters, locations and/or orientationsfor each of these different ultrasound probes in a workflow of aprocedure and automatically return to those different parameters,locations and/or orientations during different stages of the workflow.It is further envisioned that embodiments of the present system maysupport guidance to the locations and/or orientations of each of thesedifferent ultrasound probes in the workflow of a procedure and/orprovide guidance to a robotic guidance system to automatically returnthe different ultrasound probes to corresponding locations and/ororientations, wherein one or more of the probes have a differentlocation and/or orientation for one or more of the views. It is furtherenvisioned that an alternate tracking modality such as electromagnetictracking (EM), InSitu, optical tracking, etc., may be employed inaddition to, or in conjunction with, FORS to act as a guidance system(e.g., GPS) and simplify navigation and re-navigation of one or moreultrasound probes during a workflow of a procedure.

Accordingly, embodiments of the present system may provide for a systemand method in which finding and reclaiming salient views (e.g.,standardized and/or personalized for the patient) of a surgical scenemay performed. The system may provide intuitive instructions to a userso that the user may readily and easily place an ultrasound probe in adesired position and/or orientation to obtain ultrasound information atthe desired position and/or orientation. The system may simplifyvisualization parameters and may remember these parameters such asultrasound probe position, orientation, ultrasound image gain, depth,etc., and may set these parameters even across disjoint subsystemsand/or probes. Accordingly, embodiments of the present system may employOSS methods and the like to record and/or recall desired views in anultrasound-guided procedure.

Examples of Ultrasound Related Portions of One or More Embodiments

As a clinician may attempt to navigate devices and effect therapy at asurgical site, the clinician may need to see the surrounding surgicalsite from multiple viewpoints in order to ensure proper care. Forexample, the clinician may need to adjust ultrasound settings and/orparameters such as gain, compression, contrast, depth, etc., for eachview to obtain optimal image quality. Embodiments of the present systemmay incorporate the combination of ultrasound and OSS methods to assistthe clinician in recording and recalling ultrasound settings, with thefollowing variants: upon a user-initiated command (e.g., a buttonclick), the system may read and store all relevant ultrasound settings(e.g., parameters), along with associated probe position(s) and/ororientation(s) as for example by OSS. In accordance with embodiments ofthe present system, these settings may then be recalled so that theclinician may restore a saved view at a later time. The system mayfurther generate an overlay for example over a stored image of an areaof interest, of a virtual ultrasound probe(s) indicating the properposition(s) and/or orientation(s) for one or more of the views.

Further, upon automatic detection of a salient view in an ultrasoundimage, embodiments of the present system may alert a clinician of thisand may provide for example a user interface for the clinician to finetune the probe position and/or orientation, as well as ultrasoundsettings and/or parameters as desired. Such an alert may be because theultrasound image is registered to the patient anatomy, so a part of theanatomy that is in the view of the ultrasound is roughly known. Thisknowledge may be used in combination with image segmentation techniqueswhich may be known in art and are not discussed further for the sake ofclarity. One or more portions of operating parameters, locations, etc.,may then be saved and subsequently recalled, as is discussed herein.

It is also envisioned that two or more recorded views (e.g., storedultrasound images of multiple views, such as two, three, four, fiveviews, . . . , up to all views for a given procedure) may be presentedsimultaneously to aid a clinician in evaluating a surgical scene,without the need to reposition the ultrasound probe or further adjustultrasound settings and/or parameters. Since both the images and theultrasound probe are registered to the patient anatomy (e.g., to thepatient workspace), the images may be shown as separate views and/orprovided together in a single image in a proper anatomical context(e.g., within a single image with the positioning based on theanatomical positioning provided by each view). Further, in cases wheresequences of views are well established for a procedure, the system mayassist and instruct the clinician on the steps to acquiring all desiredviews. These views may be stored as a library of views, or generatedpreoperatively by the clinician. These views may then be stored and forma part of a workflow for a procedure and may be recalled by the systemand/or user when desired.

Examples of Procedure Related Portions of One or More Embodiments

A surgical procedure may entail acquisition of common anatomicalviewpoints; the combination of OSS methods to track an ultrasound probeduring a specific surgical procedure may be harnessed as follows. Thesystem may determine whether ultrasound probe is at a desired view(e.g., at a proper position and/or orientation) using the OSS methods totrack the ultrasound probe. Upon automatic detection of a desiredultrasound probe position and/or orientation (e.g., at a desired view),the system may alert a user (such as a clinician) of such and providefor the user to manually tune the ultrasound settings and/or parameters.Such an alert may be possible because the OSS methods may be registeredto the patient anatomy, and because in many cases the desired views maybe known a priori. The complete set of parameters and/or settings andthe corresponding ultrasound information for each view may then bestored in a memory of the system as a view library of a process workflowand subsequently recalled during the process workflow as discussedherein. It is further envisioned that in embodiments of the presentsystem, stored desired standard views for a process workflow may bedisplayed as an augmented reality visual rendering, indicating to theuser all of the views to be collected for proper evaluation of thesurgical scene (e.g., with the views put together into a single image orotherwise positioned in proper anatomical positions) during a futureprocess workflow. In accordance with embodiments, the views may bepresented in an order following the procedure work flow and/or may beordered/re-ordered as desired.

Examples of Shape-Sensing Related Portions of One or More Embodiments

In accordance with one or more embodiments, a virtual overlay of theprobe may be rendered to indicate to the user (e.g., a clinician, etc.)where the ultrasound probe should be placed in order to acquire theviews desired/needed to perform a surgical procedure. For example, theshape of one or more contiguous portions of the ultrasound probe (e.g.,up to all of the probe or inserted portion) may be rendered, thus,providing to the user richer information on how the ultrasound probeshould be positioned and/or oriented to acquire a desired view or views.

Full shape information (such as of an inserted portion) may be renderedand may be particularly useful when manual guidance of an ultrasoundprobe is used. Unlike a robotic manipulator which may readily manipulatean ultrasound probe to a desired position and/or orientation, a humanoperator (e.g., a clinician, etc.) may find the same task challengingdue to the unintuitive hand-eye coordination caused by having tocognitively map disjoint coordinate systems. Accordingly, full shapeinformation rendering of the ultrasound probe, which may be otherwiseredundant, provides information (e.g., using a graphics display, etc.)to the human operator about the context of the probe position and/ororientation. Furthermore, since the probe may be tracked using OSSmethods, the image or probe coordinate system may be registered to theuser's point-of-view, thus unifying the disjoint coordinate system andsimplifying manual guidance of the ultrasound probe during a processworkflow. In a robotic procedure, a user may finalize a position and/ororientation obtained by the robotic procedure to adjust the position toa final position and/or orientation as desired.

FIG. 7 shows a portion of a system 700 in accordance with embodiments ofthe present system. For example, a portion of the present system mayinclude a processor 710 (e.g., a controller) operationally coupled to amemory 720, a user interface (UI) including a rendering device such as adisplay 730, sensors 740 such as a shape sensing device (SSD), one ormore ultrasound probes 750, and a user input device 770. The memory 720may be any type of device for storing application data as well as otherdata related to the described operation. The application data and otherdata are received by the processor 710 for configuring (e.g.,programming) the processor 710 to perform operation acts in accordancewith the present system. The processor 710 so configured becomes aspecial purpose machine particularly suited for performing in accordancewith embodiments of the present system. The operation acts may includeconfiguring a system by, for example, a registration system inaccordance with system settings. The operation acts may also include theprocessor 710 obtaining a workflow from the memory 720 including desiredviews and associated parameters, such as parameters related to settingsfor the one or more ultrasound probes 750 and the sensors 740 includingcorrelation information that correlates a positon of the sensors 740 inrelation to the ultrasound probe 750 (e.g., provides offset informationrelating a position of the sensors 740 to the position of the ultrasoundprobe (750)).

The processor 710 may control one or more tracking systems (e.g., thesensors 740) such as utilizing a FORS tracking method so that sensorinformation signals indicative of a location of the sensors 740 may begenerated. The processor 710 may process received signals such as sensorinformation, transform these signals to location signals, and maygenerate content which may include image information (e.g., still and/orvideo images including ultrasound image information), data, parameters,positions, orientations, guidance information and/or graphs that may berendered on, for example, a UI of the system such as on the display 730,a speaker, etc. The content may include image information as may begenerated by a medical imaging system of the present system, guidanceinformation, etc. Further, the content may then be stored in a memory ofthe system such as the memory 720 for later use. Thus, operation actsmay include requesting, providing, and/or rendering of the content. Theprocessor 710 may render the content such as video information on a UIof the system such as a display of the system. The processor 710 maydetermine and render the UI of the system such as on the display of thesystem.

The user input device 770 may include a keyboard, a mouse, a trackball,or other device, such as a touch-sensitive display, which may be standalone or part of a system, such as part of a personal computer, apersonal digital assistant (PDA), a mobile phone (e.g., a smart phone),a monitor, a smart or dumb terminal or other device for communicatingwith the processor 710 via any operable link such as a wired and/orwireless communication link. The user input device 770 may be operablefor interacting with the processor 710 including enabling interactionwithin a UI as described herein. Clearly the processor 710, the memory720, display 730, and/or user input device 770 may all or partly be aportion of a computer system or other device such as a client and/orserver.

The methods of the present system are particularly suited to be carriedout by a computer software program, such program containing modulescorresponding to one or more of the individual steps, acts, modules,etc., described and/or envisioned by the present system. Such programmay of course be embodied in a computer-readable medium, such as anintegrated chip, a peripheral device or memory, such as the memory 720or other memory coupled to the processor 710.

The program and/or program portions contained in the memory 720 mayconfigure the processor 710 to implement the methods, operational acts,and functions disclosed herein. The memories may be distributed, forexample between the clients and/or servers, or local, and the processor710, where additional processors may be provided, may also bedistributed or may be singular. The memories may be implemented aselectrical, magnetic or optical memory, or any combination of these orother types of storage devices. Moreover, the term “memory” should beconstrued broadly enough to encompass any information able to be readfrom or written to an address in an addressable space accessible by theprocessor 710. With this definition, information accessible through anetwork is still within the memory, for instance, because the processor710 may retrieve the information from the network for operation inaccordance with the present system.

The processor 710 is operable for providing control signals and/orperforming operations in response to input signals from the user inputdevice 770 as well as in response to other devices of a network andexecuting instructions stored in the memory 720. The processor 710 mayinclude one or more of a microprocessor, an application-specific orgeneral-use integrated circuit(s), a logic device, etc. Further, theprocessor 710 may be a dedicated processor for performing in accordancewith the present system or may be a particularly programmedgeneral-purpose processor wherein only one of many functions operatesfor performing in accordance with the present system. The processor 710may operate utilizing a program portion, multiple program segments,and/or may be a hardware device utilizing a dedicated or multi-purposeintegrated circuit. Embodiments of the present system may provideimaging methods to acquire and/or reconstruct images. Suitableapplications may include imaging systems such as ultrasound images.However, without limitation it should be understood that embodiments ofthe present system may further include imaging systems such as MRI,(computer-aided tomography (CAT), optical, X-ray, and/or combinationsthereof. Further, embodiments of the present system may be ideallysuited for surgical interventional techniques which may generate andrender image and/or sensor information from one or more imaging systems(e.g., ultrasound, CT scans, MRI, etc.) having different coordinatesystems wherein results are provided (e.g., images, position and/ororientation information) in real-time with a unified coordinate system.

Further variations of the present system would readily occur to a personof ordinary skill in the art and are encompassed by the followingclaims. Finally, the above-discussion is intended to be merelyillustrative of the present system and should not be construed aslimiting the appended claims to any particular embodiment or group ofembodiments. Thus, while the present system has been described withreference to exemplary embodiments, it should also be appreciated thatnumerous modifications and alternative embodiments may be devised bythose having ordinary skill in the art without departing from thebroader and intended spirit and scope of the present system as set forthin the claims that follow. In addition, any section headings includedherein are intended to facilitate a review but are not intended to limitthe scope of the present system. Accordingly, the specification anddrawings are to be regarded in an illustrative manner and are notintended to limit the scope of the appended claims.

In interpreting the appended claims, it should be understood that:

a) the word “comprising” does not exclude the presence of other elementsor acts than those listed in a given claim;

b) the word “a” or “an” preceding an element does not exclude thepresence of a plurality of such elements;

c) any reference signs in the claims do not limit their scope;

d) several “means” may be represented by the same item or hardware orsoftware implemented structure or function;

e) any of the disclosed elements may be comprised of hardware portions(e.g., including discrete and integrated electronic circuitry), softwareportions (e.g., computer programming), and any combination thereof;

f) hardware portions may be comprised of one or both of analog anddigital portions;

g) any of the disclosed devices or portions thereof may be combinedtogether or separated into further portions unless specifically statedotherwise;

h) no specific sequence of acts or steps is intended to be requiredunless specifically indicated;

i) the term “plurality of” an element includes two or more of theclaimed element, and does not imply any particular range of number ofelements; that is, a plurality of elements may be as few as twoelements, and may include an immeasurable number of elements; and

j) the term and/or and formatives thereof should be understood to meanthat only one or more of the listed elements may need to be suitablypresent in the system in accordance with the claims recitation and inaccordance with one or more embodiments of the present system.

What is claimed is:
 1. A surgical guidance system, comprising: a memory;a rendering device; an ultrasound probe; a shape-sensing-device (SSD)associated with the ultrasound probe and having a predetermined positionand orientation with regard to the ultrasound probe; and a controllercoupled to the memory, the rendering device and the SSD, the controllerbeing configured to: determine at least one of a location and anorientation of the ultrasound probe based upon position sensorinformation (PSI) received from the SSD, select a view of a plurality ofviews of a workflow that are stored in the memory; select a proceduretype from a workflow table stored in the memory, obtain view settinginformation (VSI) for the selected view from the memory, the VSIcomprising parameters and at least one of a position and an orientationof the ultrasound probe for each of the plurality of views, determineguidance information based upon a comparison of the determined at leastone of the location and the orientation of the ultrasound probe and theat least one of a position and orientation of the ultrasound probe forthe selected view, and render the determined guidance information on therendering device and set ultrasound probe parameters based on theparameters of the VSI for the selected view.
 2. The surgical guidancesystem of claim 1, wherein the SSD comprises at least one positionsensor and the controller is further configured to query the at leastone position sensor to obtain the PSI, the PSI being indicative of atleast one of a position and orientation of the at least one positionsensor relative to a workspace.
 3. The surgical guidance system of claim1, wherein the controller is further configured to render the determinedguidance information as directional indications to guide the ultrasoundprobe to at least one of a position and orientation of the ultrasoundprobe for the selected view.
 4. The surgical guidance system of claim 1,wherein the controller is further configured to display simultaneouslyon the rendering device two or more of the plurality of views of theworkflow.
 5. The surgical guidance system of claim 4, wherein thecontroller is further configured to display on the rendering device anindication for each of the two or more of the plurality of views.
 6. Thesurgical guidance system of claim 4, wherein the controller is furtherconfigured to display on the rendering device the two or more of theplurality of views as a single view with the two or more of theplurality of views anatomically positioned.
 7. The surgical guidancesystem of claim 1, wherein the controller is coupled to at least onetransducer of the ultrasound probe to obtain ultrasound imageinformation using the set ultrasound probe parameters.
 8. The surgicalguidance system of claim 7, wherein the controller is further configuredto reconstruct an image based upon the ultrasound image information inaccordance with the set ultrasound probe parameters.
 9. The surgicalguidance system of claim 1, wherein the controller is further configuredto associate and store two or more of current parameters, location,orientation, and ultrasound information of the ultrasound probe inassociation with the selected view.
 10. The surgical guidance system ofclaim 1, wherein the controller is further configured to determine whichof the plurality of views is closest to the determined at least one oflocation and orientation of the ultrasound probe and select the viewbased on which view is determined closest.
 11. The surgical guidancesystem of claim 1, wherein the controller is further configured to uselandmarks to guide the ultrasound probe.
 12. The surgical guidancesystem of claim 1, wherein the controller is further configured toselect the view based on a threshold.
 13. A method for guiding anultrasound probe for obtaining ultrasound information, the methodcomprising: determining at least one of location and orientation of theultrasound probe based upon position sensor information (PSI); selectinga view of a plurality of views of a workflow that are stored in amemory; selecting a procedure type from a workflow table stored in thememory, obtaining view setting information (VSI) for the selected view,the VSI comprising information related to parameters and at least one ofa position an orientation of the ultrasound probe for each of the views;determining guidance information based upon a comparison of the at leastone of the location and the orientation of the ultrasound probe and theat least one of a position and orientation of the ultrasound probe forthe selected view; and outputting the determined guidance informationand setting ultrasound probe parameters based on the parameters of theVSI for the selected view.
 14. The method of claim 13, wherein themethod further comprises an act of querying at least one position sensorof a shape-sensing-device (SSD) to obtain the PSI, the PSI beingindicative of at least one of a position and orientation of the at leastone position sensor relative to a workspace.
 15. The method of claim 13,wherein the outputting of the determined guidance information comprisesacts of generating guidance instructions corresponding to the determinedguidance information and rendering the generated guidance instructionson a rendering device.
 16. The method of claim 13, wherein theoutputting comprises outputting the determined guidance information asdirectional indications to guide the ultrasound probe to at least one ofa position and orientation of the ultrasound probe for the selectedview.
 17. The method of claim 13, further comprising using landmarks toguide the ultrasound probe.
 18. The method of claim 13, furthercomprising selecting the view based on a threshold.
 19. A non-transitorycomputer readable medium comprising computer instructions which, whenexecuted by a processor, configure the processor to: determine at leastone of location and orientation of an ultrasound probe based uponposition sensor information (PSI); select a view of at least oneregistered view of a workflow; select a procedure type from a workflowtable stored in the memory, obtain view setting information (VSI) forthe selected view, the VSI comprising information related to parametersfor each of the registered views and at least one of a position andorientation of the ultrasound probe for each of the views; determineguidance information based upon a comparison of the at least one of thelocation and the orientation of the ultrasound probe and the at leastone of a position and orientation of the ultrasound probe for theselected view; and output the determined guidance information andsetting ultrasound probe parameters based on the parameters of the VSIfor the selected view.
 20. The non-transitory computer readable mediumof claim 19, wherein the instructions, when executed by the processor,further cause the processor to query at least one position sensor of ashape-sensing-device (SSD) to obtain the PSI, the PSI being indicativeof at least one of a position and orientation of the at least oneposition sensor relative to a workspace.
 21. The non-transitory computerreadable medium of claim 19, wherein the instructions, when executed bythe processor, further cause the processor to display simultaneously twoor more of the plurality of views of the workflow.
 22. Thenon-transitory computer readable medium of claim 21, wherein theinstructions, when executed by the processor, further cause theprocessor to display simultaneously the two or more of the plurality ofviews as a single view with the two or more of the plurality of viewsanatomically positioned.
 23. The non-transitory computer readable mediumof claim 21, wherein the instructions, when executed by the processorfurther cause the processor to generate guidance instructionscorresponding to the determined guidance information and render thegenerated guidance instructions on a rendering device.
 24. Thenon-transitory computer readable medium of claim 21, wherein theinstructions, when executed by the processor further cause the processorto output the determined guidance information as directional indicationsto guide the ultrasound probe to at least one of a position andorientation of the ultrasound probe for the selected view.
 25. Thenon-transitory computer readable medium of claim 19, wherein theinstructions, when executed by the processor, further cause theprocessor to use landmarks to guide the ultrasound probe.
 26. Thenon-transitory computer readable medium of claim 19, wherein theinstructions, when executed by the processor, further cause theprocessor to select the view based on a threshold.